This application is based on Application No. HEI 9-261268 filed in Japan, the content of which is hereby incorporated by reference.
1. Field of the Invention
The present invention relates to a display device for displaying images, for example, to generate a distance image in real space relative to the viewpoint of an observer in augmented reality (AR), and to display an AR image based on a generated distance image.
2. Description of the Related Art
The art of influencing human perception to present an environment as if in the real world by presenting computer-generated information to a person is known as virtual reality (VR). The art of virtual reality allows a person to enter and experience an virtual environment as if in the real world. Virtual reality is also known as artificial reality.
Augmented reality (hereinafter referred to as xe2x80x9cARxe2x80x9d) is one area in the field of virtual reality technology. In the field of augmented reality, real space and computer-generated virtual space are mixed and presented to the viewer. In augmented reality, real space is enhanced by virtual space.
In AR technology the purpose is to heighten the quality of real space by suitably arranging an virtual environment in a real environment, whereas the purpose of VR technology is to completely immerse a person in a virtual environment. The presentation of a image as an virtual environment is visual AR. Hereinafter AR refers to to as xe2x80x9cvisual AR.xe2x80x9d
Examples of the uses considered for AR technology include systems which superimpose the image from a patient""s brain CT scan on the actual brain of the patient during surgery, systems supporting work such as assembly and repairs and the annotation of parts of an actual mechanical device by computer graphics, systems which evaluate the impression of how furniture will look in an actual room, and systems which evaluate sights at actual locales and at actual size when urban redevelopment programs build new buildings.
AR technology typically uses a head-mounted display (HMD) to present both actual space and virtual space to a person simultaneously. Head-mounted displays include those using optical systems and those using video signals.
FIG. 13 illustrates the principle of AR system 80 using a conventional optical system type head-mounted display, and FIG. 14 illustrates the principle of AR system 90 using a conventional video signal type head-mounted display. In AR system 80 shown in FIG. 13, a real space image is transmitted through a half mirror 81 arranged on the line of sight in front of the eyes of an observer so as to be viewed by the observer, and a virtual space image projected by a projector 82 is viewed via half mirror 81 so as to be viewed by the observer. In this way a virtual space image overlaps a real space image. A position/posture sensor 83 is provided to detect the observation point of the head of an observer; the signals of a virtual space image are transmitted from a workstation 84 to projector 82 based on the output of the position/posture sensor 83.
In AR system 90 shown in FIG. 14, a real space image is projected by double mirror 91 to two image sensors 93 arranged at conjugate positions relative to the right and left eyes of an observer. The projected real space image and a virtual space image are combined by workstation 95, and the composite image is projected by projector 92. As a result, the composite image of the combined real space image and the virtual space image is presented to the observer via double mirror 91.
To realize augmented reality (AR), there is, in addition to the conventional problem of specifying positional matching of the images, the requirement of accurately expressing the screening relationship. In both the aforesaid AR system 80 and 90, there must be accurate expression of the screening relationship wherein space and objects far from the observer are screened by space and objects near the observer.
In the former AR system 80, real space is visible behind objects in virtual space, such that an observer sees both the images of virtual space and real space simply overlapped. In this system, however, the screening relationship cannot be expressed completely.
In the latter AR system 90, there is no distance relationship and objects in virtual space typically hide objects in real space, such that a screening relationship cannot be accurately expressed. That is, although the screening relationship is accurate when objects in virtual space are nearer than objects in real space, objects in virtual space block objects in real space such that the objects in real space appear far away even when they are at near positions, and the screening relationship is reversed.
According to opinion in perceptual psychology, in the state wherein a screening relationship cannot be accurately manifested, stereo vision is obstructed and unexpected stereo vision results due to the contradiction between vergence-parallax information and the screening relationship. In other words, accurate expression of the screening relationship is extremely important in the art of augmented reality.
The present applicants have come up with a head-mounted display capable of accurately expressing screening relationship. In this head-mounted display, the distance of a virtual space image and the distance of a real space image are compared relative to the pixels of the virtual space image and pixels of the real space image at the same position so as to select and display the pixels nearer to the observer.
FIG. 15 is a perspective view of the exterior of the previously mentioned head-mounted display (HMD) 70.
In HMD 70, slit projectors 71R and 71L are respectively arranged at positions near the right and left eye of an observer to measure the distance to an observation object from the viewpoint of the observer. The reflected light of slit light UR and UL projected from the two slit projectors 71R and 71L is received by photoreceptors 73R and 73L disposed at each image sensor 72R and 72L of HMD 70, so as to obtain accurate distance information of each pixel of a the right and left two-dimensional images (real space images). The screening relationship can be accurately expressed based on the right and left distance information.
In the HMD 70 of FIG. 15, however, the slit projectors 71R and 71L must be positioned near the right and left eye, respectively thereby complicating the construction of the disposition and mounting, and enlarging the overall head-mounted device 70.
In particular, since the HMD 70 is an active type device, photoreceptors 73R and 73L are required which are separate from the photoreceptor device for inputting the two-dimensional images so as to receive the slit light UR and UL projected from slit projectors 71R and 71L, thereby making it difficult to accurately match the lines of sight.
An object of the present invention is to eliminate the previously described disadvantages.
A further object of the present invention is to provide a distance image generating device capable of increasing the freedom of arrangement of a distance detector, and capable of obtaining distance information which matches the line of sight of an input two-dimensional image based on distance information from a different position than the line of sight of the input two-dimensional image by generating other necessary distance information based on distance information measured by a single distance detector.
A still further object of the present invention is to provide a compact head-mounted device capable of obtaining binocular distance information from distance information measured by a distance detector provided at a position near one eye.
The invention itself, together with further objects and attendant advantages, will be best understood by reference to the following detailed description taken in conjunction with the accompanying drawings.